The present invention relates to a method and an apparatus for a dynamic loop and post filtering. The method or apparatus for the dynamic loop and post filtering of decoded video is applicable to multimedia and video applications. In particular it is useful for the delivery of video over low bandwidth networks such as Internet where the picture quality is badly affected by a coding artefact such as blocky noise.
Filtering is a very common signal processing technique that is used to process a picture to enhance or change its appearance. Forms of the filtering can be broadly classified into two categories. They are picture enhancement and picture restoration. This disclosure mainly deals with the latter case of picture restoration. Degradation to the picture is introduced by the compression technique. In particular, the most common type of the degradation is a blocky artefact that results from high compression in a block-based transform-coding technique.
FIG. 1 shows a typical block-based transform codec (coder-decoder). As shown in FIG. 1, an encoder and a decoder form a synchronized pair. The encoder emulates decoding operations in a local decoder. The typical encoder comprises a block discrete cosine transform (DCT) module 11, a quantization (Q) module 12, a variable length coding (VLC) module 13, an inverse quantization (IQ) module 14, an inverse DCT (IDCT) module 15, a frame store module 16 and a motion compensation module 17. A picture is passed into the encoder where it is partitioned into blocks. Apart from the non-predicted picture (normally referred to as I-Pictures or Intra picture), the blocks undergo motion compensation where the previous decoded picture is subtracted from the current block. These blocks are then passed to the DCT module 11 where they are transformed into the DCT domain. The transform coefficients are then quantized by the Quantization module 12 and entropy-coded in the VLC module 13 before being sent to the decoder. The encoder also emulates the operations of the decoder by performing an inverse quantization step on the quantized coefficients and then transforming the coefficients back into the spatial domain. At this point, the motion compensated prediction, if it was used earlier, is added to the block to form the reconstructed picture. The reconstructed picture is stored in the frame store 16 where it is used by the motion compensation module 17 for the prediction of the next picture.
The typical corresponding decoder comprises of a variable length decoding (VLD) module 18, an inverse quantization (IQ) module 19, an inverse DCT (IDCT) module 20, a frame store module 21 and a motion compensation module 22. Noticed that apart from the VLD module 18 that performs the entropy decoding, the remaining modules 19 to 22 are identical to the modules 14 to 17 of the local decoder in the encoder. These modules perform the same function as described above.
The blocky artefact mentioned above is caused by the quantization step where noise is added to crucial low and high frequency components. This results in discontinuities at the block boundary of the picture, which shows up as blocky noise.
There are several forms of blocky noise removal filters. The first is a simple form of post filter 210, as shown in FIG. 2. This form of filter is placed at the output of the video decoder and only affects the picture being displayed. The second form of filter is a loop filter. This is placed in the motion compensation loop of the encoder and decoder. There are two places where the filter may be placed in the loop. The first is after the motion compensation block 310, as shown in FIG. 3. The second is before the frame store 410, as shown in FIG. 4. The first case is used in codecs such as the ITU-TH 261 standard. In this disclosure, the term of loop filter will refer to the second case where the filter is placed before the frame store 410.
In the case of the post filter, the filter affects only the output display picture. There is no propagation effect because the filtering effect is not stored in the frame memory. In the loop filter, there is a propagation effect of the filter because the filtered picture is stored into the frame memory and is used for the motion compensation of the subsequent picture.
Currently, the most effective filters used for filtering blocky noise uses knowledge that the discontinuities appear only at specific locations of the picture, namely at the block boundary. This allows the filter to remove the blocky noise without introducing excessive smoothing to the detail of the picture.
At high bit rates, where the motion compensation residual is coded by the transform coefficients, the post filtering technique is effective. The blocky noise present in the frame store is repaired by the transform coefficients being sent at subsequent pictures. So only the output picture need to be filtered. Putting the filter into the loop would actually cause propagation effect and reduce the overall picture quality.
At low bit rates, where the motion compensation residual is not completely coded, the loop filtering technique is more effective. This is because the blocky noise present in the frame store is not repaired by the transform coefficients being sent in subsequent pictures. Therefore, in the post filter case, the propagated blocky noise is not removed especially in the case where motion compensation has shifted the discontinuities to positions other than the block boundary.
Therefore, the first problem to be solved is how to integrate two types of filters since each of the filters is only effective at different ends to the bit rate distribution.
The second problem is how to keep the bit rate down in the low bit rate case when the loop filter is removing picture detail from the frame store.